Discussion of “Gibbs sampling, exponential families, and orthogonal polynomials” by Diaconis, Khare, and Saloff-Coste

It is our pleasure to congratulate the authors (hereafter DKSC) on an interesting paper that was a delight to read. While DKSC provide a remarkable collection of connections between different representations of the Markov chains in their paper, we will focus on the “running time analysis” portion. This is a familiar problem to statisticians; given a target population, how can we obtain a representative sample? In the context of Markov chain Monte Carlo (MCMC) the problem can be stated as follows. Let Φ = {X0, X1, X2, . . .} be an irreducible aperiodic Markov chain with invariant probability distribution π having support X and let Pn denote the distribution of Xn|X0 for n ≥ 1, ie, Pn(x,A) = Pr(Xn ∈ A | X0 = x). Then, given ω > 0, can we find a positive integer n∗ such that ‖Pn(x, ·)− π(·)‖ ≤ ω (1) where ‖ · ‖ is the total variation norm? If we can find such an n∗, then, since ‖Pn − π‖ is nonincreasing in n, every draw past n∗ will also be within ω of π, thus providing a representative sample if we keep only the draws after n∗. There is an enormous amount of research (too much to list here!) on this problem for a wide variety of Markov chains. Unfortunately, there is apparently little that can be said generally about this problem so that we are forced to analyze each Markov chain individually or at most within a limited class of models or situations. This is somewhat reflected in the current paper since, as noted by DKSC, the techniques introduced here do not apply to even all of the exponential families (with a conjugate prior) in the paper. However, DKSC derive some impressive results that would seem difficult to improve upon. In the rest of this discussion we will review some of their findings and compare them to results possible via the so-called (by DKSC) “Harris recurrence techniques.”